ECCUMENT RESUME 



EE 050 173 



TM OCO 570 



A U 1 HO R 

inn 

i h SlllUiio 
PUE DATE 
NOTE 



t a fi p E e 1 1 r P ci u i t • ; Tools, Joan S 

Let i jut ion and lieasui errant in tne Attective D c era i n : 
Apprcciatic n o : h u ira n Ac c c p 1 1 s n m** n i: s . 

lennsylvani a State Dept, ot E d u c a i <: n , Hanishurq. 

F e t 71 

let,; taper pi*:; onto cl at t i. o Annual Meeting ot the 
American :aucaticnal ’.♦"‘search Association, New York, 
Now York, February 1V71 



E l i- S P : I 0. E 
D ? S C r, I i ' e S 



II - 1 N T I? Ir RS 



r. T> r, S P r 1 C t K r - $ 0 • b b EC- $ ^ . kb 

Elective behavior, Factor Analysis, * Grade b, 
^draue 11, asuroraent reel: n j gue s , I ictcual 
St i lull, Reliability, ^Hesoaich Tools, best 
C o n s t r uc 1 1 c i i , Visual Measures 

P e n j I s y 1 va n t a 1 n v e n t c r y o t Cultural A j. p i e c i a 1 1 o n s , 
^Pennsylvania Plan, 1ICA, Things E topic Eo 
Itvuiiuory, 7FD 



A I SIT- AC k 



The Him tnret levels ot tn«j taxo.ony in t h e 
a tree five domain grided the development ot tve in vejitorj.es-- 1 he 
Pennsylvania inventory cl Cultural Appreciations ( P I c n) tor 11th 
u i aders and Things People Do (FPL) tor 5th graders-- to .measure 
appreciation ct hup an aceca p lishnc nt s in seven areas: politics, 
sciences, sports, literature, visual arts, music, and theatre* Alp! a 
coot i ic lent ic l the PICA is .5*, and tor the TPE, .75. i. no tog ra phs 
are use i in tic itD. With a sanpie ct i,U00 at each grade level, 
analyses oi the instruments included comparison ot nran.s, analysis ot 
correlations, tact or analysis and analysis or cumulative structure. 
T ine rypo* to si zed hierarchical structure is supported ty the TED data 
hut not i,y the PICA data. Many inhere r cos and implications are 
discussed. (Authoi/GS) 
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Obje ct ive 

One of the stated goals of education in Pennsylvania is that pupils 
should ahow appreciation for human accomplishments in the natural sciences, 
the social sciences, the humanities arid the arts. The Bureau of Educational 
Quality Assessment In the Pennsylvania Department of Education was given 
the assignment to measure the degree to which pupils are showing these 
appreciations, in order to assess the level of pupil achievements, measuring 
inventories were needed. A search for existing inventories proved fruitless. 
Instrumentation in the area of cultural appreciations reflects the lack of 
attention traditionally paid this area in the curriculum. A further limita- 
tion was imposed by the corresponding lack of attention traditionally paid 
the affective domair. in the measurement field. 

The objective of this paper i£. to describe the process of defining and 
measuring in the affective domain through the development of two inventories - 
Pennsylvania Inventory of Cultural Appreciations (PICA) for 11th graders and 
Things People Do (TPD) for 5th graders. 

Method 

PICA was developed first. Human accomplishments were divided into seven 
catcgor ies : 
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Politics 
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Visual Arts 
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Sciences 
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Music 
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Sports 

Literature 
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The first three levels of the Taxonomy of Educational Objectives ; Af f ec t ive 
Domain (Krathwohl, Bloom, Masia, 1956) were used to guide the construction of 
items. Items were developed to measure in each of the seven categories at each 
of the first three levels of the taxonomy. 

Level 1: Receiving (Attending) At this level the learner 

is expected to be willing to receive or attend to 
a certain stimulus rather than to avoid it. 

Sample Items — If you had the opportunity, would you like 
to: Visit an art museum? See a soccer game? 

Attend an opera? 

Level 2: Responding At this level the learner is expected 

to move from merely receiving a stimulus to actively 
attending. Also at this level not only does the 
learner actively attend but begins to gain satis- 
faction in responding. Music becomes an emotional 
involvement. Politics become a zealous pursuit. 

Reading becomes a vicarious exparience. 

Sample Items -- (1) Poetry is best when I can read it aloud to 
myself. (2) Most works of art are too difficult to 
understand. (3) I would like to be able to vote in 
a state or national election. 

Level 3: Valuing At this level the learner is expected to 

display a certain behavior with sufficient consis- 
tency to show that he perceives the behavior to have 
worth. At this lowest level of valuing, commitment 
is still uncertain. The learner continues to 
reevaluate his beliefs and his position is still 
tentat ive . 

Sample Items — (I) Most scientists ere interested only in 

machines, not in people. (2) Most people in public 
life are dedicated to helping the public. (3) Tax 
money should be used to support the theater. 

In early September of 1968, seventy-nine items were tried out with five 
Ilth grade groups: an ''average" ability group from an urban school (N = 23); 

the "highest" and "lowest" ability groups from a rural school (N ■ 59); and 
an "honors" group and a "general" group from a suburban school (N «=■ 54). The 
total sample was 136 Ilth grade students. Post-test individual and group 
interviews were held to collect student opinions about the clarity or ambiguity 
~f the items, the appropriateness of the response options and any other reactions. 
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As a result of the tryout, seemingly ambiguous items were rewritten, 

Imbalance was corrected with an equal number of items for each of the seven 

categories and nondiscriminating items were eliminated or revised. The format 

was refined and the first 21 items were designated '’knowledge' 1 rather than 

affective items. The resulting PICA contains 77 items? 

21 Knowledge items 
21 Receiving items 
21 Responding items 
14 Valuing items 

PICA was then administered to a group of urban 5th grade pupils to test 
its applicability at the lower grade level. The results indicated that the 
vocabulary and the level of comprehension required were too advanced for many 
5th grade pupils. Things People Do was then constructed for 5th grade pupils. 

TPD attempts to measure pupil appreciations in the same seven categories 
according to the first three levels of the taxonomy. Since reading level is 
of utmost concern for 5th grade pupils, a group of photographs is used to 
depict each of the seven categories. The inventory asks the pupil to consider 
a group of four pictures and chen to answer three questions about the pictures. 
For example, in the music category the pictures depict a symphony orchestra, 
ballet dancers, a rock group and a vocalist. The pupil is asked: 

1. Would you like to see or hear one or more of these 
kinds of musical activities? (Receiving) 

2. Would you like to take part in one or more of 
these kinds of musical activities? (Responding) 

3. Do you think music is very important, somewhat 
important or not important? (Valuing) 

TPD contains 21 items: 

7 Receiving items 
7 Responding items 
7 Valuing items 
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In the fall of 1969, the inventories were administered to 20,000 5th 
graders and 20,000 11th graders as part of a statewide assessment battery. 

The results from these large samples were used to develop norms for both 
inventories . 

Subsamples of approximately 3,000 at each grade level were drawn in order 
to analyze the technical properties of the inventories. The subsamples are 
representative of pupils from districts of varying degrees of wealth and 
proportionately representative of pupils from schools of varying sizes. 

Analysis and Re sults 

Reliability . The reliability estimates of inventories such as PICA and 
TPD present several problems for which only partial solutions exist. A test- 
retest coefficient would be most appropriate for this analysis if the necessary 
conditions could be met (i.e., minimal practice effect and/or the existence 
of parallel forms and the availability of the sample for retesting). These 
conditions could not be met at the present stage of inventory development. 

Overall estimates of coefficient alpha (Cronbach, 1951) were, therefore, computed 
for both TPD and PICA. It is recognized that this statistic may provide an 
unprecise estimate of reliability if hierarchiality exists. The coefficients 
can be considered a conservative estimate of reliability under the assumption 
that the strata are fixed rather than random. The strata vere defined as fixed 
(Rabinowitz and Eikeland, 1964). The obtained alpha values for the TPD and 
PICA are .79 and .92, respectively. 

The inventories were designed to represent a hierarchy of affective 
behaviors associated with human accomplishments in the sciences, arts and 
humanities. The empirical confirmation of the existence of this hierarchy was 
approached in several ways. 
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Compar i son of Means * The first analysis considered differences among 
strata means. The concept of level of involvement is used to serve the same 
function in the affective domain as the concept of level of complexity serves 
in the cognitive domain (Kropp and Stoker , 1966). The theory of an affective 
hierarchy requires that valuing represent a higher level of involvement than 
responding, which in turn is higher in involvement than receiving. It is, 
therefore, hypothesized that the mean score for receiving will be highest, 
for responding less high and for valuing least high. Evidence of such ordering 
is present in the first two levels of TPD, but is reversed in PICA (See 
Table 1). 

Analysis of Correlations . An analysis of the inter-item correlation 
matrix analogous to the Guttman Radex Technique (Guttman, 1964) was undertaken 
with TPD. If the hierarchy exists, then it is expected that inter-item cor- 
relations will be greatest among items within levels, next in magnitude between 
items in adjacent levels and least in magnitude between items in more remote 
levels. Although evidence for the hierarchy exists in the TPD data, the 
correlations do not completely fit the hypothesized order. Two phenomena are 
present. First, the inter-itera correlations within levels are significantly 
higher than the inter-item correlations across either adjacent or remote 
levels as expected. However, the order of magnitude of the mean inter-item 
correlations between adjacent and between remote levels are reversed from the 
hypothesized order (See Table 2). Second, there appears to be more item 
variance common within categories of activity across levels than within 
levels across categories of activities. The mean inter-item correlation 
within activity categories is .47. The disparate nature of the activities 
represented probably accounts for this phenomenon and also serves to suppress 
somewhat the emergence of the hierarchical structure. 



This situation is further reinforced when the pattern of responses to 
individual items within each category is examined. In two of the seven 
categories (politics and science) the mean item response for valuing is higher 
than for the first two levels, which is a reversal of the hypothesized 
direction. In the remaining five categories (music, art, literature, theater 
and sports) the means f ollow the inter-item correlation pattern predicted by 
the hierarchy (See Table 3). 

Several possible explanations are considered for the deviations from 
the pattern. The hierarchy may overlap at the responding and valuing levels. 

A person may need to value an activity before he can respond, thereby 
confirming the overall correlational pattern between these levels. Further- 
more, the items may not have elicited in two categories the appropriate 
responses from a substantial number of students. A more likely explanation 
is that politics and science deviated so sharply from the pattern that its 
existence in the correlation matrix is submerged. 

A similar analysis was conducted with PICA. Because the patterns of mean 
item responses suggested an interaction between categories of activity and 
levels of the hierarchy, a modification of the analysis procedure was under- 
taken. lti addition to the sets of within levels, adjacent levels and remote 
levels inter-item correlations, the set of inter-item correlations which 
were specific to each activity category was removed and treated as a separate 
category. The hypothesized order of magnitude of the average correlation for 
each of these sets was as follows: 

r within activities > r within levels * r adjacent levels ? r remote levels 

As in the case of the TPD analysis, there is a significant difference 
among the mean correlations. The order in this instance is exactly congruent 
with the hypothesized order (See Table A). It would appear then that if this 
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pattcrn holds up In subsequent data collections, the existence of an affective 
hierarchy in the area of appreciation of human accomplishments is supported. 

The definition, however, of the levels of the hierarchy by the measuring instru- 
ment presents some problems. These will be discussed in a later section. 

Factor A nalysis . The inventories were factor analyzed as part of a. 
battery and as separate inventories. The results support TPD as a separate 
inventory apart from inventories to measure citizenship, creativity, inter-personal 
relationships and interest in school. The factor analysis of TPD as a 
separate inventory produced a principal components solution with a general 
factor upon which each item has a factor loading of at least .40. A varimax 
rotation produced four factors which accounted for 54% of the explained 
variance. Factor one included items from art, theater and music. Factor two 
included items from science and literature. Factors three and four are 
defined by items from politics and sports in that order. The three levels of 
the taxonomy - receiving, responding and valuing - are not discernible in the 
factor structure. 

The factor analysis for the 11th grade instrument, PICA, also supported 
Its use as a separate measure from others included In the total battery. All 
Items have a loading of at least .53 In the general factor from the principal 
components solution. Three factors were defined by the varimax rotation. They 
explained a total of 60% of the variance. Factor one accounted for theater, 
art and valuing Items. Factor two Included sports, politics and science 
items. Factor three was defined by music and a general set of receiving 
items. The hypothesized literature items were represented among all three 
factors. Two levels of the affective hierarchy seem to be emerging as 
separate factors In this analysis. For more detailed information on this aspect 
of the study see Beers, 1971. 
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Ana lysis of Cumulative Struc ture. Additional analyses were designed to 
test the hypo thea ized property of cumulat iveuess, The theory of hlerarchial ity 
requires that in order to value one must respond and in order to respond one 
must receive. Corollary to this hypothesis is the expectation that the 
highest level of involvement, valuing, will be less frequently represented 
in the population than responding, which* in turn, will be of a lesser fre- 
quency than receiving. To test these hypotheses the patterns of response to 
the items on the tvo inventories (TPD and PICA) were organized into sets 
according to the degree of consistency between the pattern and the hierarchy. 
Strong consistent patterns wore those which showed a high level of receiving, 
a lower level of responding and the lowest ltve] of valuing. This set is 
consistent with the expected order of frequencies of the affective levels in 
the population. The weak consistent set included those patterns of response 
which were not contradictory in the serse that a person valued without being 
willing to receive. A pattern which indicated that a student was moderately 
willing to receive, to respond and to value would be assigned to this set. The 
inconsistent set of patterns included those which showed high levels of valuing, 
moderate levels of responding and an unwillingness to receive. In order to 
exhaust the possibilities of response patterns to the instrument, all items 
were assigned to a triad consisting of receiving, responding and valuing items. 
The directionality of the triad was examined by considering the difference 
between the mean of the first pair of the triad and the mean of the second 
pair. If the fir3t mean was larger than the second, then the item was 
considered to be supportive of the hierarchy and assigned to the strong set. 

If the means of the two pairs were equal, then the triad was considered 
consistent and assigned to the weak set of response patterns. If the mean 
of the second pair was larger than the mean of the first pair, then the triad 
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This conclusion is further borne out by an additional analysis. A panel of 

four judges independently assigned each item on the PICA to the three levels 

receivej respond and value. There only one instance of disagreement on 

$ 

the 21 receiving items. In contrast there were only 6 out of 21 responding 
items upon which there was agreement and 11 of 14 valuing items. A 
major problem appears to be the definition of responding items as distinct 
from valuing items. T<r '~ 



Summary 

In summary the researchers conclude that the development and analysis 

of TPD and PICA demonstrate several things. 

Reliable measurements in the hard-to-measure 
affective area can be constructed. 

Photographs can successfully be used to elicit 
responses from elementary pupils. 

The levels of the taxonomy can form a theoretical 
basis for developing inventories in the affective 
domain . 

The levels of the rexonomy emerge as distinct when 
several kinds of statistical techniques are applied. 

The hypothesized hierarchical structure of the 
taxonomy appears to exist with TPD for 5th graders. 

A hierarchical structure appears also to exist 
with PICA for 11th gradern, but not in the 
hypothesized direction. 

The work on defining operationally the hierarchical structure ac the 
11th grade must be considerably extended. Continuous development and tryout 
of both instruments with new samples are underway. 
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Table 1 



Mean Item Responses 



Inventories 


Levels. 


Receiving 


Responding 


Valuing 


TPD 








Grade _ f . 








N = 
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2. .*7 
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2.46 
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